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ABSTRACT 


The purpose of this thesis is to examine which auditory cues need to be included 
in a virtual representation of a Close Quarters Combat Room Clearing Operation. Future 
missions of the United States Anned Forces, especially those of the Army and Marine 
Corps, are increasingly likely to be conducted in cities or built-up areas. A critical need 
exists for MOUT (Military Operations in Urban Terrain) training by our anned forces, 
and the entire spectrum of military training needs to be addressed. Doctrine and 
principles, classroom instruction, drills, live fire exercises, as well as live and virtual 
simulation all have a part to play in developing an acceptable level of expertise in 
MOUT. The way in which training tasks are represented in a VE, including which audio 
cues must be presented, needs to be explored. This study will examine the auditory cues 
used during a MOUT mission and how those cues should be presented in a virtual MOUT 
scenario. 

This thesis uses the Critical Decision Method of knowledge elicitation to obtain 
an auditory Critical Cue Inventory from room clearing subject matter experts. Those 
experts will then conduct a virtual room clearing operation using the game America’s 
Army: Operations, and compare the auditory cues presented in the game scenario to the 
real-world execution of the task. 
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I. INTRODUCTION 


Future missions of the United States Anned Forces, especially those of the Anny 
and Marine Corps, are increasingly likely to be conducted in cities or built-up areas. 
Well-trained and well-equipped Infantry forces are the key component for military 
success in urban environments (Sulzen, 1997). There has been a great deal of research 
over the past several years to better understand the challenges of Military Operations in 
Urban Terrain (MOUT). Such operations place great demands on military personnel for 
new types of technical skill. They also generate a peculiar set of cognitive requirements 
related to a wide range of judgments and decisions necessary in urban settings. Our 
military must be prepared for MOUT because the U.S. is a victim of its own successes. 
While our technical superiority will act as a deterrent for adversaries to engage us in 
conventional warfare, urban conflict offers attractive advantages to these potential 
adversaries. Urban warfare results in the potential loss of civilian life, destruction of 
important landmarks, ease of access for media coverage, and increased potential to inflict 
injuries on our soldiers with relatively unsophisticated weapons systems. In addition, the 
U.S. military has spent less time preparing for MOUT than for conventional warfare 
(Phillips, et al. 2001). As the likelihood of MOUT operations increases, the amount of 
training by U.S. forces to conduct these operations must also increase. 

Obviously, a critical need exists for MOUT training by our anned forces, and the 
entire spectrum of military training needs to be addressed. Doctrine and principles, 
classroom instruction, drills, live fire exercises, as well as live and virtual simulation all 
have a part to play in developing an acceptable level of expertise in MOUT. Virtual 
Reality (VR) technology is beginning to have a large impact on military training. Units 
are no longer required to spend scarce resources (time and money) in preparation for and 
travel to large-scale training events and exercises. Soldiers are frequently able to train in 
simulators and experience many of the same effects as they would had they actually gone 
to the field. However, although sophisticated simulations can enhance military training, 
the technology still does not offer a veridical representation of the real world. There is an 
ongoing effort to detennine exactly which individual and collective military tasks can be 
represented adequately in simulation (Salter, Eakin, & Knerr 1999). In addition to 
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determining which tasks to train using VEs, the VE must contain accurate perceptual 
stimuli and cues, including auditory cues. 

Immersion in a VE has been defined as the “sense of actually being present in the 
virtual scenario” (Vince, 2001). Although there are many different methods to increase a 
user’s sense of presence or immersion, among the most important are the sensory cues 
that are presented to the user. The greater the number of sensory inputs provided to 
different modalities, the greater the sense of presence (Lessiter, Freemen, & Davidoff 
2001). Audio is a primary emotion-inducing medium in movie presentation and is said to 
comprise at least 50% of the motion picture experience (Holman, 2000). As the 
immersive qualities of a VE increase, the ability of VE participants to feel like they are 
actually present in the environment also increases. In most VEs, visual cues represent the 
majority of the sensory cues that lead to increased immersion. However, auditory cues 
have also been shown to play a crucial role in immersion (Sanders & Scorgie 2002). It is 
critical to determine exactly which auditory cues need to be presented in a training 
system or VE, and at what level of fidelity. Inversely, one needs to make certain that 
auditory cues that are not actually present in the environment are not placed in the VE. 
All auditory cues presented must be accurate and appropriate. 

Why is it important to develop an accurate model of the auditory cues associated 
with a room clearing operation? In order to successfully conduct room clearing, all 
members of the team must achieve and maintain a high level of situational awareness. 
Situational awareness is the degree to which one is able to maintain a common operating 
picture of all aspects of the tactical situation. This picture includes an understanding of 
the friendly and enemy situation and the urban battle space. Since units will have to 
conduct operations in changing mission environments, it is imperative for commanders 
and leaders at all levels to achieve and maintain the best possible degree of situational 
awareness. Enhanced situational awareness will enhance lethality, survivability, and 
operational tempo (Department of the Army, 2002). The way that soldiers develop this 
common operating picture, or situational awareness, is by mentally combining all of the 
environmental cues received through their various senses. 

Obviously, visual cues are critically important for creating an accurate mental 
image of one’s environment. In certain circumstances, however, the auditory aspect of a 
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situation can be equally, if not more important, in understanding everything that is going 
on around you. Situational awareness in a room clearing operation can be characterized 
by one’s understanding of two key aspects: 1) the enemy situation, and 2) the status of 
the friendly force. The room clearing team must locate the enemy in order to kill them, 
and must also prevent the enemy from locating and determining the mission and intent of 
the friendly team. When walls and other obscurants block the visual cues that would lead 
to this information, auditory cues become the primary means of developing situational 
awareness. 

The room clearing team will use any auditory cues available to help develop a 
mental image. Any sounds that the enemy makes inside a room, footsteps, weapon 
noises, voices, whispers, rattling of equipment, will help the team members understand 
the enemy situation that they are about to face. Those same sounds, however, if made by 
the friendly force and heard by the enemy, can easily compromise the friendly mission 
and intent, and can lead to mission failure. An accurate virtual representation of a room 
clearing task must present these auditory cues in an accurate and appropriate way. 

Some of the auditory cues associated with a real-world execution of a room 
clearing task are extremely loud. Weapons firing, explosions, grenades detonating, and 
people yelling will all have an impact on the people conducting the task. The effect of 
those loud noises must be examined and modeled in the virtual task representation. 
Tinnitus is a perceived ringing or some other kind of sound that is referred to the ear and 
cannot be traced to any external stimulus (Wever, 1949). This condition may be acute 
and temporary, as the result of excessive stimulation by a loud sound (Stevens & Davis, 
1938). Temporary Threshold Shift (TTS) occurs after exposure to a loud noise, when the 
muscles of the middle ear contract in an effort to protect the ear from noise damage. This 
TTS temporarily degrades a person’s ability to hear more subtle sounds after exposure to 
a very loud sound (Pickles, 1982). Again, an accurate virtual representation of a room 
clearing task must present these auditory cues in an accurate and appropriate way. 

But how does one determine exactly what is accurate and appropriate for 
representing auditory cues in a room clearing operation? Obviously, in order to obtain 
insight into the real-world execution of the task, and what sounds are actually used by 
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experts in the room clearing domain, a knowledge elicitation technique involving real- 
world experts will have to be developed and utilized. 

America’s Army: Operations (AA:0) is a desktop-based videogame/virtual world 
representing life as a soldier in the United States Army. One of the levels in AA:0 
represents a mission that involves conducting a Close Quarters Battle (CQB) Room 
Clearing Operation in a MOUT setting. This MOUT scenario depicts a team of 
dismounted infantry soldiers entering a building in which some of the rooms are occupied 
by enemy soldiers. The friendly team must eliminate all hostile forces in the building, 
while protecting any non-combatants that may also be present. The findings of this study 
will be used as an analysis of auditory cues that need to be included in virtual 
representations of room clearing tasks. 

A. THESIS OVERVIEW 

This thesis will examine which auditory cues are associated with the perfonnance 
of an actual Close Quarters Battle Room Clearing Operation, determine which of those 
cues need to be included in a virtual representation of that task, and explore the potential 
negative training transfer issues associated with incomplete or unrealistic auditory cues 
that may be presented in a virtual training environment. 

1. Research Questions 

• Which auditory cues need to be modeled in a Virtual Environment Close 
Quarters Battle Room Clearing Operation? 

• What sound cues are associated with the performance of the actual task? 

• Will the lack of certain auditory cues result in a negative training transfer 
from the virtual scenario to the real-world execution of the task? 

2. Discussion 

In order to make a virtual CQB room clearing operation as realistic and 
immersive as possible, an accurate description and model of the auditory cues associated 
with the real-world performance of the task must be made. Exclusion of key auditory 
cues or over-emphasis of non-critical auditory cues in the virtual scenario could result in 
reduced immersion and negative transfer of training to the real world execution of the 
task. Creating an accurate model of the auditory cues involved in the task is essential to 
preclude these problems. 
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The CQB room clearing task is extremely complex and typically of very short 
duration. Recognition of which auditory cues are actually used immediately prior to and 
during the task is critical to the development of an accurate auditory model. The end 
product of this thesis will be a Critical Cue Inventory (CCI) of all the auditory cues 
associated with a CQB room clearing operation. The CCI will present those auditory 
cues in a rank-ordered list from most important to least important for each subtask of the 
overall room clearing task. Once the auditory CCI of the real-world room clearing task is 
completed, it will be used to conduct an analysis of the auditory cues that are being 
presented in the America’s Anny: Operations representation of that task. In order to 
create the CCI, this thesis will use the Critical Decision Method to elicit domain specific 
knowledge from subject matter experts (SMEs) in the room clearing domain. These 
SMEs will be students at the Naval Postgraduate School who have developed a high 
degree of expertise in room clearing operations. The term expert generally refers to an 
individual who has over ten years experience and would be recognized as having 
achieved proficiency in the subject domain (Klein, Calderwood, & MacGregor, 1989). A 
structured interview that is created using the framework of the Critical Decision Method 
can provide specific feedback as to the auditory cues associated with each task and 
subtask of a given behavior. The CCI will be developed using the knowledge elicitation 
results obtained during the SME interviews. The CCI will provide a rank-ordered list of 
the auditory cues that need to be considered for inclusion in any virtual representation of 
the room clearing scenario. 

After the Auditory Critical Cue Inventory is created, it will be used as a baseline 
for evaluating the auditory cues that are currently being presented in the room clearing 
scenario of AA:0. Some of the same SMEs who were interviewed to create the CCI will 
use it to document the auditory cues that they would expect to hear during a specific 
room clearing operation and then compare their expectations to the auditory cues that are 
actually presented as they conduct that operation in AA:0. Any differences between 
expectations of auditory cues in a real-world situation and auditory cues presented in the 
virtual situation will be examined for the potential to create a negative training transfer 
effect. 
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B. THESIS ORGANIZATION 

This thesis is organized into the following chapters: 

• Chapter I: Introduction. This chapter provides an overall outline of the 
thesis and describes the research objectives and motivation behind this 
research 

• Chapter II: Background. This chapter reviews current U.S. doctrine 
concerning Military Operations on Urban Terrain (MOUT) and the 
auditory cues associated with room clearing operations, knowledge 
acquisition techniques, structuring techniques, and the Critical Decision 
Method. 

• Chapter III: Subject Matter Expert Interview. This chapter describes the 
process used to recruit and interview room clearing subject matter experts, 
and presents the Auditory Critical Cue Inventory that resulted from the 
interviews. 

• Chapter IV: Evaluation of Room Clearing Scenario. This chapter 
describes the procedures used to evaluate the auditory cues presented in a 
room clearing scenario in America’s Army: Operations. 

• Chapter V: Conclusions and Recommendations. This chapter provides a 
summary of the procedures used in this study and of the recommendations 
for the auditory cues to be presented in virtual room clearing scenarios. It 
also suggests potential future work in this area. 

• Appendices: 

A. Interview. 

B. Mission Briefing and Maps. 

C. Participant Consent Fonns. 
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II. BACKGROUND 


A. MOUT TASK AND ASSOCIATED SOUND CUES 

Before utilizing the Critical Decision Method to fonnulate a structured interview 
centered on any task, a cursory understanding of the task must be achieved. The task 
must be analyzed, understood, and broken down into sub-tasks that can be used as a 
framework for further investigation. The primary reference for MOUT issues in the U.S. 
Army and Marine Corps is FM 3-06.11 Combined Arms Operations In Urban Terrain 
(fonnerly FM 90-10-1 An Infantryman’s Guide To Combat In Built-Up Areas). FM 3- 
06.11 defines Urban Operations (UO) as operations planned and conducted in an area of 
operations that includes one or more urban areas. An urban area consists of a 
topographical complex where man-made construction or high population density is the 
dominant feature. FM 3-06.11 defines Urban Combat as those offensive and defensive 
operations that include a high density of Infantry-specific tasks. Urban combat operations 
are conducted to defeat an enemy on urban terrain who may be intermingled with 
noncombatants. Engagements in a UO scenario are typically characterized as extremely 
rapid and violent. Engagement ranges are close. Studies and historical analyses have 
shown that only 5 percent of all targets are more than 100 meters away. About 90 
percent of all targets are located 50 meters or less from the identifying soldier. Few 
personnel targets will be visible beyond 50 meters and they usually occur at 35 meters or 
less. Engagement times are short. Enemy personnel present only fleeting targets. Unless 
combat has been taking place in an urban area for an extended period of time, units will 
encounter large numbers of noncombatants. Noncombatants may be encountered during 
offensive operations as a result of clearing buildings and city blocks or when preparing 
for defensive operations. 

MOUT scenarios present unique requirements for extreme clarity in situational 
awareness but also severely degrade the individual soldier’s ability to use his own 
physical senses to enhance that situational awareness. Situational awareness is the degree 
to which one is able to maintain a common operating picture of all aspects of the tactical 
situation. Smoke, darkness, fog, rain, snow, ice, and glare make it hard to see. The 
extended wear of night vision goggles, protective masks, or laser protective lenses causes 
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stress. Intense noise not only causes stress by itself, but it further isolates the soldier 
from human contact and interferes with situational awareness. To the company level 
leaders situational awareness means being able to answer certain questions: 

1. Where am I (in respect to the urban area or my assigned sector)? 

2. Where are my soldiers? What is their current status/activity? 

3. Where are friendlies (adjacent and supporting units)? What is their current 
status/activity? 

4. Where is the enemy? What are the enemy’s capabilities? (Department of the 
Army, 2002) 

Clearly, auditory cues, especially in limited visibility situations requiring 
enhanced situational awareness, play a vital role in leaders and soldiers being able to 
answer these questions (Department of the Army, 2002). 

Successful combat operations in urban areas depend on the proper employment of 
the rifle squad. Each member must be skilled in moving, entering buildings, clearing 
rooms, employing hand grenades, selecting and using fighting positions, navigating in 
urban areas, and camouflage (Department of the Army, 2002). The main emphasis of 
this study will be on the room clearing phase of Urban Operations. 

There are two different techniques for conducting a room clearing operation. The 
High Intensity technique, in which a fragmentation or concussion grenade is thrown into 
a room before the US forces enter, and the Precision Technique. The primary 
consideration in determining which technique to use is whether or not there is a 
possibility of noncombatants being present in the room. Precision clearing techniques do 
not replace other techniques currently being used to clear buildings and rooms during 
high-intensity combat. Precision room clearing techniques are used when the tactical 
situation calls for room-by-room clearing of a relatively intact building in which enemy 
combatants and noncombatants may be intermixed. They involve increased risk in order 
to clear a building methodically, rather than using overwhelming firepower to eliminate 
or neutralize all its inhabitants (Department of the Army, 2002). 

From a conceptual standpoint, standard high-intensity room clearing drills can be 
thought of as a deliberate attack. The task is to seize control of the room with the 
purpose being the neutralization of the enemy in the room. The fragmentation and or 
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concussion grenades can be thought of as the preparatory fires used before the assault. 
As in a deliberate attack against any objective, the assaulting elements move into position 
using covered and concealed routes. The preparatory fires (fragmentation and or 
concussion grenades) are initiated when soldiers are as close to the objective as they can 
get without being injured by the fires. The assault element follows the preparatory fires 
onto the objective as closely as possible. A rapid, violent assault overwhelms and 
destroys the enemy force and seizes the objective (Department of the Army, 2002). 

Compared to the deliberate attack represented by high-intensity room clearing 
techniques, precision room clearing techniques are more conceptually like a 
reconnaissance in force or perhaps an infiltration attack. During a reconnaissance in 
force, the friendly unit seeks to determine the enemy’s locations, dispositions, strength, 
and intentions. Once the enemy is located, the friendly force is fully prepared to engage 
and destroy it, especially if surprise is achieved. The friendly force retains the options of 
not employing preparatory fires (fragmentation and or concussion grenades) if they are 
not called for (the enemy is not in the room) or if they are inappropriate (there are 
noncombatants present also). The attacking unit may choose to create a diversion (use a 
stun grenade) to momentarily distract the defender while they enter and seize the be 
planned and executed with care. Units must train, practice, and rehearse precision room 
clearing techniques until each fire team and squad operates smoothly. Each unit member 
must understand the principles of precision room clearing: surprise, speed, and controlled 
violence of action (Department of the Army, 2002). 

Surprise is the key to a successful assault at close quarters. The fire team or 
squad clearing the room must achieve surprise, if only for seconds, by deceiving, 
distracting, or startling the enemy. Sometimes stun grenades may be used to achieve 
surprise. These are more effective against a nonalert, poorly trained enemy than against 
alert, well-trained soldiers (Department of the Army, 2002). 

Speed provides a measure of security to the clearing unit. It allows soldiers to use 
the first few vital seconds provided by surprise to their maximum advantage. In precision 
room clearing, speed is not how fast you enter the room, rather it’s how fast the threat is 
eliminated and the room is cleared (Department of the Anny, 2002). 
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Controlled violence of action eliminates or neutralizes the enemy while giving 
him the least chance of inflicting friendly casualties. It is not limited to the application of 
firepower only, but also involves a soldier mind-set of complete domination. Each of the 
principles of precision room clearing has a synergistic relationship to the others. 
Controlled violence coupled with speed increases surprise. Hence, successful surprise 
allows increased speed (Department of the Army, 2002). 

In order to examine the auditory cues associated with the room clearing task, one 
must first gain an understanding of the procedures used to actually conduct the task. 
MAJ Warren Aronson has recently completed a Cognitive Task Analysis of the entire 
room clearing task that describes, in detail, the steps used to complete a room clearing 
operation (Aronson, 2002). For a detailed examination of the auditory requirements, an 
overview of these procedures and fundamentals can be used. 

The eleven fundamentals of precision room clearing address actions soldiers take 
while moving along confined corridors to the room to be cleared, while preparing to enter 
the room, during room entry and target engagement, and after contact. Team members: 

1. Move tactically and silently while securing the corridors to the room to be 
cleared. 

2. Carry only the minimum amount of equipment. (Rucksacks and loose items 
carried by soldiers tire them, slow their pace, and cause noise.) 

3. Arrive undetected at the entry to the room in the correct order of entrance, 
prepared to enter on a single command. 

4. Enter quickly and dominate the room. Move immediately to positions that allow 
complete control of the room and provide unobstructed fields of fire. 

5. Eliminate all enemy in the room by fast, accurate, and discriminating fires. 

6. Gain and maintain immediate control of the situation and all personnel in the 
room. 

7. Confirm whether enemy casualties are wounded or dead. Disarm, segregate, and 
treat the wounded. Search all enemy casualties. 

8. Perfonn a cursory search of the room. Determine if a detailed search is required. 

9. Evacuate all wounded and any friendly dead. 
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10. Mark the room as cleared using a simple, clearly identifiable marking in 
accordance with the unit SOP. 

11. Maintain security and be prepared to react to more enemy contact at any moment. 
Do not neglect rear security (Department of the Army, 2002). 

The following is a list of required actions for performing the task of clearing a 
room: 

1. The squad leader designates the assault team and identifies the location of the 
entry point for them. 

2. The squad leader positions the follow-on assault team to provide overwatch and 
supporting fires for the initial assault team. 

3. Assault team members move as close to the entry point as possible, using 
available cover and concealment. 

a. If an explosive breach or a ballistic breach is to be perfonned by a 
supporting element, the assault team remains in a covered position until 
the breach is made. They may provide overwatch and fire support for the 
breaching element if necessary. 

b. All team members must signal one another that they are ready before the 
team moves to the entry point. 

c. Team members avoid the use of verbal signals, which may alert the enemy 
and remove the element of surprise. 

d. Assault team members must move quickly from the covered position to 
the entry point, minimizing the time they are exposed to enemy fire. 

4. The assault team enters through the breach. Unless a grenade is being thrown 
prior to entry, the team should avoid stopping outside the point of entry. 

a. The number 2 man may throw a grenade of some type (fragmentation, 
concussion, stun) into the room before entry. 

b. The use of grenades should be consistent with the ROE and building 
structure. The grenade should be cooked off before being thrown, if 
applicable to the type of grenade used. 

c. If stealth is not a factor, the thrower should sound off with a verbal 
indication that a grenade of some type is being thrown (“frag out,” 
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“concussion out,” “stun out”). If stealth is a factor, only visual signals are 
given as the grenade is thrown. 

5. On the signal to go, or immediately after the grenade detonates, the assault team 
moves through the entry point and quickly takes up positions inside the room that 
allow it to completely dominate the room and eliminate the threat. Unless 
restricted or impeded, team members stop movement only after they have cleared 
the door and reached their designated point of domination (Department of the 
Army, 2002). 

The first man enters the room and eliminates the immediate threat. He has the 
option of going left or right, normally moving along the path of least resistance to one of 
two corners. The direction each man moves in should not be preplanned unless the exact 
room layout is known. Each man should go in a direction opposite the man in front of 
him. Every team member must know the sectors and duties of each position. On order, 
any member of the assault team may move deeper into the room overwatched by the 
other team members. Once the room is cleared, the team leader signals to the squad 
leader that the room has been cleared (Department of the Army, 2002). 

When conducting precision clearing, soldiers are very close to each other as they 
engage targets. The high volume of noise makes communications extremely difficult. 
The command and control techniques used during precision combat must consist of terms 
and actions that soldiers are familiar with and to which they know how to respond. The 
use of verbal commands and signals within the assault element are extremely important. 
The soldier must always let others in the assault element know where he is and what he is 
doing. As an example, terms similar to the ones listed in Table 1 should be a part of each 
soldier’s vocabulary IAW unit SOP (Department of the Army, 2002). Each unit with a 
room clearing mission will have its own SOP of verbal commands. These commands are 
practiced and rehearsed during each training exercise that the unit conducts (Aronson 
2002). Table 1 shows a full representation of the types of commands that units need to 
incorporate into their verbal command SOPs. 
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“STATUS!” 

Signal by an element leader that requires all members to 
report whether their sectors are clear and if they are prepared 
to continue the mission. 

“CLEAR!” 

Signal given by individuals to report their sector is clear. 

“UP!” 

Signal given by individuals to report they are ready to 
continue the mission (weapon loaded, equipment accounted 
for). 

“ROOM CLEAR!” 

Signal from team leader to team members, squad leader, and 
follow-on teams that the room is secure and cleared. 

“COMING OUT!” 

Signal given by an individual or team that they are about to 
exit a room. 

“COME OUT!” 

Reply given by security element or follow-on team that it is 
safe to exit the room. 

“COMING IN!” 

Signal given by an individual who is about to enter an 
occupied room. 

“COME IN!” 

Reply given by an occupant of a room stating it is safe to 
enter. 

“COMING UP (DOWN)!” 

Signal given by an individual or team that is about to ascend 
or descend a stairway. 

“COME UP (DOWN)!” 

Reply given by security element that it is safe to ascend or 
descend a stairway. 

“MAN DOWN!” 

Signal given when an individual has been wounded or injured 
and cannot continue his mission. 

“SHORT ROOM!” 

Signal given by either the number 1 man or the number 2 man 
to indicate a small room, and that all team members should 
not enter. 

“GRENADE!” 

A command given by any soldier, when an enemy grenade has 
been thrown. All soldiers need to take immediate actions. 
Although difficult, the soldier should identify the location of 
the grenade, if possible. 

“GO LONG!” 

A command given by one member of the team to tell another 
team member to take up security farther into the room or 
farther down a hallway. 

“GUN DOWN” 

A signal given when an individual’s weapon has 
malfunctioned and is being corrected. 

“GUN UP” 

A signal given when an individual has corrected a malfunction 
and is ready for action. 

“RELOADING” 

A signal given when an individual is reloading any weapon 
system. This signal is followed by “GUN UP” when ready. 


Table 1. Typical Verbal Commands 

The use of loud verbal commands may reveal to the enemy the location and 


immediate intent of friendly forces. Although code words may be substituted, they can be 
heard and used by enemy forces if friendly forces use them too loudly (Department of the 
Army, 2002). 

Urban areas provide unique target acquisition challenges to units. Buildings mask 

movement and the effects of direct and indirect fires. The rubble from destroyed 
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buildings, along with the buildings themselves, provides cover and concealment for 
attackers and defenders, making target acquisition difficult. Urban areas often favor the 
defender’s ability to acquire targets so this makes offensive target acquisition extremely 
important, since the side that fires first may win the engagement. Target acquisition must 
be continuous, whether a unit or soldier is halted or moving. The six steps of target 
acquisition, search, detection, location, identification, classification, and confirmation are 
no different in an urban environment than anywhere else but are usually performed at a 
much faster pace (Department of the Army, 2002). 

Using all senses during the search step enhances the detection capabilities of all 
soldiers on the urban battlefield. Soldiers searching the urban battlefield for targets 
should employ target acquisition devices. Observation duties must be clearly given to 
squad members to ensure 360 degrees and three-dimensional security as they move. This 
security continues at the halt. Soldiers soon recognize the sights, smells, sounds and so 
forth, associated with their urban battlefield and can soon distinguish targets. Stealth 
should be used when moving in urban areas since there are often short distances between 
attackers and defenders. Hand and arm signals should be used until contact is made. The 
unit should stop periodically to look and listen. Routes should be carefully chosen so that 
buildings and piles of rubble can be used to mask the unit’s movement. Personnel, 
weapons, and vehicles have distinguishing signatures. Soldiers must recognize signatures 
so they can acquire and identify targets. This is extremely important in the urban 
battlefield, where one or more senses can be degraded. For example, soldiers operating 
in an urban area where smoke is used as an obscurant will have their sense of sight 
degraded, since they may not be able to see through the smoke with the naked eye. Their 
sense of smell and breathing is also affected. Running engines, vehicles, and soldiers 
moving through rubble-covered streets can be heard for great distances. Vehicles driven 
in urban areas produce more noise than those moving through open terrain. Soldiers 
moving through rubble on a street or in the halls of a damaged building create more noise 
than in a wooded area. Sounds and smells can aid in acquiring targets at night since they 
transmit better in the cooler, damper night air. Dust and noise created by the firing of 
some weapons such as a ta nk main gun can be seen and smelled. Voices can often be 
heard at long distances, with the sound reflecting off of structures. As a minimum, 
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identification must detennine if a potential target is friend, foe, or, a noncombatant. 
Correct identification is the key to preventing fratricide. To determine an appropriate 
method of dealing with a target, the soldier must detennine the danger it represents. It 
requires quick decisions as targets are observed and occurs virtually simultaneously with 
identification. Situational awareness is vitally important. Multiple targets must be 
classified from most dangerous to least dangerous and engaged starting with the most 
dangerous. The rapid verification of the initial identification and classification of the 
target is the final step of target acquisition. Identification, classification, and confirmation 
are done simultaneously (Department of the Army, 2002). 

MOUT and room clearing operations present many additional unique challenges 
in regard to auditory cues for soldiers. Command, control, communications, and the 
limited visibility afforded to soldiers all have auditory aspects that require examination. 
The first phase of the attack should be conducted when visibility is poor. Troops can 
exploit poor visibility to cross open areas, gain access to rooftops, infiltrate enemy areas, 
and gain a foothold. If the attack must be made when visibility is good, units should 
consider using smoke to conceal movement. Commanders and leaders must consider the 
effect that city lights, fires, and background illumination have on night vision devices. 
These elements may limit the effectiveness of night vision goggles (NVGs) and make 
thennal imagery identification difficult. Control of the urban battlefield is difficult. In 
urban areas, radio communications are often less effective than field telephones and 
messengers. Units often fight without continuous communications. Pyrotechnic signals 
are hard to see because of buildings and smoke. The high noise level of battles within 
and around buildings degrades voice alerts. Voice communication can also signal the 
unit’s intention and location to the enemy. Radio communications in urban areas pose 
special problems to tactical units. Communications equipment may not function properly 
because of the massive construction of buildings and the environment. In addition to the 
physical blockage of line of sight transmissions, there is also the interference from 
commercial power lines, absorption into structures and the presence of large quantities of 
metal in structures. Also, the noise of urban combat is much louder than in other areas, 
making sound or verbal signals difficult to hear. Urban operations require centralized 
planning and decentralized execution. Therefore, effective vertical and horizontal 
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communications are critical. Leaders must trust their subordinates’ initiative and skill, 
which can only occur through training. The state of a unit’s training and cohesion are 
vital, decisive factors in the execution of operations in urban areas. Visual signals may 
also be used but are often not effective because of the screening effects of buildings, 
walls, and so forth. Signals must be planned, widely disseminated, and understood by all 
assigned, attached, or OPCON units. Increased noise makes the effective use of sound 
signals difficult. Verbal signals may also reveal the unit’s location and intent to the 
enemy (Department of the Army, 2002). 

B. KNOWLEDGE ACQUISITION 

There are many experts that possess a great store of knowledge in the subject 
domain of MOUT room clearing and that have actual experience using the related 
auditory cues necessary for a successful room clearing operation. In order for an 
experimenter who is not a domain expert to use that knowledge, however, a method of 
acquiring and representing domain specific knowledge must be developed and utilized. 
Expert knowledge may be obtained from many different sources: textbooks, reports, data 
bases, case studies, empirical data, and personal expertise. The primary source, however, 
is the domain expert, the individual with expertise in the field of interest. This 
knowledge must usually be obtained through direct interaction with the expert. The 
actual elicitation of knowledge is highly problematic because experts possess much 
information that is cognitively complex, pragmatic, and tacitly formulated (Patterson, 
1990). 

Any expert has knowledge that is explicit and objective, as well as knowledge that 
is more implicitly fonnulated (Hawkins, 1983). The later is usually very difficult for 
experts to articulate (Broadbent, Fitzgerald, and Broadbent, 1986). It is important to 
recognize that a domain expert will possess knowledge of several different kinds (Berry, 
1987). Each type of knowledge demands a technique that can most effectively capture it. 
The technique must transform this knowledge to a representation suitable for the 
inference strategy used in the problem solving technique. Rather than use a single 
knowledge acquisition technique, several techniques should be employed, with each 
matched to a different kind of knowledge (Gammack and Young, 1985). 
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The nature of the task is an important feature that should be explored when the 
components of an expert’s knowledge are investigated. The nature of the task is salient 
in that it detennines the possible strategies an expert uses to complete or solve a task 
(Hogarth, 1974). When the particular problem solving task is isolated and identified, the 
type of knowledge necessary to solve that problem, independent of any particular 
implementation, should be analyzed and described. This enables the examiner to 
decompose the expert’s compiled knowledge and to identify discrete tasks, types of 
knowledge being processed, and the relationships among the data, facts, and procedures 
(McGraw and Riner, 1987). 

Selection of the appropriate knowledge acquisition technique requires that the 
examiner recognize the type of knowledge under investigation. Major problems 
associated with expert knowledge investigation include recognition and analysis of 
domain knowledge and selection of an appropriate knowledge acquisition technique. 
One widely accepted scheme classifies knowledge into four basic types: procedural, 
declarative, semantic, and episodic (McGraw and Riner, 1987). Procedural knowledge 
includes the skills that an individual knows. It may involve an automatic response to a 
stimulus, and can be reactionary in nature. Such skills are deeply ingrained and linked 
sequentially, one step serving as the completing the next. This knowledge is implicit and 
highly compiled so that an expert will have great difficulty in both identifying and 
verbalizing it, and is therefore of primary interest to knowledge examiners (McGraw and 
Reiner, 1987). When individuals master increasingly more knowledge to carry out a task 
efficiently, they also lose awareness of what they know. This has been called the 
“paradox of expertise” (Johnson, 1983). Procedural knowledge is not necessarily motor 
in nature. This type of knowledge may also include that which is gained from implicit 
learning or an unconscious process such as socialization, perception, and the rules of 
complex games (Gammack and Young, 1985). Declarative knowledge represents surface 
level infonnation that experts can verbalize. The primary difference between this and 
procedural knowledge is the ability to verbalize or express it. Declarative knowledge is 
what the expert is conscious of knowing. Semantic knowledge represents one of the two 
theoretical types of long term memory. It reflects cognitive structure, organization, and 
representation. As a result, it will be difficult for experts to express. This type of 
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knowledge includes memories for vocabulary, concepts, facts, definitions, and 
relationships among facts. Episodic knowledge is autobiographical, experience-oriented 
information that the expert has grouped or chunked by episodes and is the second 
theoretical type of long term memory. It consists of infonnation organized by time and 
place of occurrence, and often may be described in terms of perceptual characteristics. 
This is highly compiled information and is one of the most difficult types of knowledge 
to extract and dissect. Since the knowledge is chunked, the expert may or may not be 
aware of the separate knowledge entities and decision making processes used to complete 
the task (McGraw and Reiner, 1987). 

Identification of the application task characteristics is important because this will 
influence selection of the knowledge acquisition tool and the strategies to be applied in 
building and refining the knowledge base. The characteristics of the task affect the 
manner in which an expert will store and access task-critical knowledge, and will 
detennine the problem-solving strategy. Expert knowledge is task centered, so analyzing 
the processing states and considerations an expert applies when performing a task or 
making a decision is key to attaining an initial understanding of the domain (Riesbeck, 
1984). 

Several methods have been developed for knowledge acquisition, and no single 
technique is usually used to the exclusion of others. Sometimes a combination approach 
may be used while in other circumstances different techniques may be appropriate to 
different stages of the acquisition process. When selecting a specific technique, a 
knowledge examiner should identify and isolate the problem-solving task to be simulated. 
Then the type of knowledge necessary to solve that problem should be described and 
analyzed, independent of any particular implementation (Kidd, 1987). 

The interview is the most common technique for the elicitation of domain 
knowledge from an expert (Gammack and Young, 1985). Interviews quickly allow the 
knowledge examiner to grasp important domain concepts and vocabulary. The expert 
may reveal the objects he thinks about, how they are related, the judgmental processes 
used in solving a problem, and some inference rules. Most interviews are conducted in 
an unstructured form and will seldom provide complete or well organized descriptions of 
cognitive processes (Olson and Rueter, 1987). At this point the knowledge examiner 
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should switch to focused or structured interviews that involve careful preplanning of the 
questions and their order. This represents a more goal oriented approach that may 
uncover additional data on factual knowledge, types of problems, functions of expertise, 
and explanations (Kidd, 1987). 

C. STRUCTURED INTERVIEWING TECHNIQUES 

A structured interview will be developed and administered to Subject Mater 
Experts (SMEs) in the MOUT room clearing domain in order to obtain those auditory 
cues deemed most important for inclusion in the room clearing VE. Structured 
interviews are often used in conjunction with a design that employs statistical sampling. 
This combination provides data that can be used to make projections about the entire 
population from which the sample was drawn. The steps in the evaluation design process 
are defining the questions that dictate the objectives of the study, selecting the method of 
collecting the infonnation, and preparing an analysis plan for using the collected 
information to answer the questions (U.S. General Accounting Office, 1991). 

Data-collection instruments (DCIs) are used on assignments that require the same 
or unifonn information on numerous cases. A DCI is a document containing questions 
presented in a systemic, highly precise fashion. Its purpose is to enable the evaluator to 
obtain uniform data that can be compared, summed, and, if it is quantitative, subjected to 
additional statistical analysis. An interview that uses a DCI to gather data, either by 
telephone or face to face, is a structured interview, one in which evaluators ask the same 
questions of numerous individuals in a precise manner, offering each interviewee the 
same set of possible answers (U.S. General Accounting Office, 1991). 

Designing a structured interview requires more than just writing down a set of 
questions to be asked. One must first examine the process by which the interview 
questions are identified, developed, and selected; then describe standard procedures for 
composing and formatting the questions. These procedures aim to ensure that the data 
collected are reliable and valid and to facilitate trouble-free editing and analysis of data, 
while keeping the burden on the interviewee to a minimum (U.S. General Accounting 
Office, 1991). 

The first step is to formulate the broad, overall questions to be answered by the 
evaluation. Why is the study being done? What do we hope to be able to say or prove? 
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The type of question asked will dictate the evaluation strategy. After the broad overall 
questions are developed, they must be translated into measurable elements in the form of 
hypotheses or questions. Next, the target population must be identified. The target 
population is the source level (individuals, groups, or organizations) at which the 
information is to be gathered. Then develop a pool of questions that attempt to measure 
the variables under consideration. From the pool of questions, the most useful or 
appropriate are chosen. In order for questions to be deemed appropriate, they must be 
relevant to the study, directed to the proper persons, and easily answered. Considerations 
on the type or format of question to use include how the question is delivered or 
presented, what the interviewee is asked, and available response alternatives. Among the 
types of questions asked are open-ended, fill-in-the-blank, binary-choice, and scaled- 
response. The open-ended question provides no structure for the answer, allowing the 
interviewee to discuss what he or she wishes, not necessarily what the interviewer wants 
to know. Fill-in-the-blank questions have a simple answer, usually in the form of a name, 
frequency, or amount. Binary-choice questions are the typical yes-no, true-false type of 
questions, a good format for obtaining factual information, but generally not opinions or 
feelings. In the scaled-response question, you read or show to the interviewee a scale or 
list of alternative responses that increase or decrease in intensity in an ordered fashion. 
The number of cues (scaled responses) for scaled-response questions depends on the type 
of interviewee and type of analysis expected. There is generally a physical limit to the 
number of cues to which an interviewee can react, probably around seven. In any 
interview, the order in which the questions are presented is important. Early questions, 
which set the tone for the collection procedure and can influence responses to later 
questions, also help you get to know the interviewee and to establish the rapport essential 
to a successful interview. Further methods of composing good interview questions and 
forestalling problems with comprehension or bias include considering the appropriateness 
and level of language used in the interview, the effects of qualifying language, and the 
importance of clarity. Whether interviewing language is appropriate or inappropriate 
may relate to what is said, how it is said, or when it is said. When composing interview 
questions, one should consider the level of the language used. Seek to communicate at 
the level the interviewee understands and to create a verbal setting that is conductive to 


20 



serious data gathering yet one in which the interviewee is comfortable. After composing 
an interview question, it may be necessary to use an adjective, qualifying phrase, or time 
specified to make the item complete or to give the interviewee sufficient or complete 
information. The style in which a question is couched can effect its clarity of 
communication. Style issues include such matters as question length, complexity, clutter, 
double negatives, extreme language, and defining terms. A question is biased when it 
causes interviewees to answer in a way that does not reflect their true positions on an 
issue. An interviewer may or may not be aware of the bias. Bias can appear in the stem 
(or statement) portion of the question or in the response-alternative portion. Bias may 
also result when a question carries an implied answer, choices of answer are unequal, 
“loaded” words are used, or a scaled question is unbalanced (U.S. General Accounting 
Office, 1991). 

Pretesting and expert review constitute what could be the least appreciated phase 
in the development of a structured interview, but these are perhaps the most important 
developmental steps. They constitute an iterative process that uses continuing input from 
evaluators and technical specialists to derive the final product. In pretesting, we test the 
DCI with respondents drawn from the universe of people who will eventually be 
considered for the study interviews to predict how well the DCI will work during actual 
data collection. The pretest seeks to detennine if the right questions are being asked, if 
the content of each question is relevant, and if the wording and procedures used in the 
interview are adequate. The initial steps of a pretest are the same as for actual data 
collection. Problems with the DCI or procedures often become evident immediately and 
may be dealt with then, so that the interview may proceed. Ideally, it is desirable to run 
through the entire interview without getting sidetracked. This way, you can examine the 
flow of the interview and estimate the total time needed to complete it. After the pretest, 
identify any problems that the interviewer has in asking the questions or the interviewees 
have in answering the questions. The expert review is an attempt to seek outside help on 
our approach to the structured interview. It helps to determine if the questions being 
asked and the manner in which they are asked are adequate to answer the overall question 
posed in the evaluation. By obtaining expert help, we avoid potential problems after data 
collection is complete. Persons providing expert review are not acting as interviewees. 
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They do not answer the questions but instead provide a critique (U.S. General 
Accounting Office, 1991). 

Prior to conducting a structured interview, interviewers need to be trained. 
Although they are not subject matter experts in the subject domain, they need to have at 
least a cursory understanding of the overall principles involved and any domain specific 
language that the actual SME will use. Interviewers also need to know the purpose of the 
evaluation and the procedures for conducting the interviews. At a minimum, interviewers 
need to know what an adequate answer to each question is, how to ask the questions 
correctly, not to omit questions they think are answered by other questions, and how to 
avoid introducing bias (U.S. General Accounting Office, 1991). 

Each participant in the interview (interviewer and interviewee) has a role to 
perform and a set of behaviors that assist in the performance. Because the role and 
behaviors of each one influence the conduct of the interview, they affect the conduct of 
the interview. The interviewer’s role and behaviors can be prescribed and acquired 
through training, while the interviewee’s role and behaviors must be observed by the 
interviewer. The role of the interviewer is to ask the questions, while that of the 
interviewee is to respond with answers. The interviewer must perform at least eight 
major tasks: 

1. Develop rapport with the interviewee and show interest. 

2. Give the interviewee a reason to participate. 

3. Elicit responsiveness from the interviewee. 

4. Ask questions in a prescribed order and manner. 

5. Ensure understanding. 

6. Ensure nonbias. 

7. Obtain sufficient answers. 

8. Show sensitivity to the interviewee’s burden (U.S. General Accounting Office, 

1991). 

The purpose of all the work put into designing, pretesting, revising, obtaining 
expert review, and finally administering the structured interview is to obtain data that, 
when properly analyzed, will answer the evaluation or audit question. The analysis to be 
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done will be determined to a great degree by the objectives that have been established for 
the structured interview (U.S. General Accounting Office, 1991). 

D. CRITICAL DECISION METHOD 

Since the purpose of this study is to examine the auditory cues associated with a 
MOUT room clearing operation, the structured interview administered to domain experts 
needs to be prepared using a method that will reveal those associated auditory cues. One 
such method that can generate an inventory of critical cues is the Critical Decision 
Method (CDM) (Klein, Calderwood, & MacGregor, 1989). The CDM is a retrospective 
interview strategy that applies a set of cognitive probes to actual nonroutine incidents that 
required expert judgment or decisionmaking (Klein, Calderwood, & MacGregor, 1989). 
It relies on interviews with proficient decisionmakers to examine recent cases of interest 
and determine the basis for situation assessment and decisionmaking during those 
nonroutine incidents (Klein, Calderwood, & MacGregor, 1989). 

A common element that exists in all efforts to improve human performance is a 
specification of the bases of skill performance that will enable task perfonnance to be 
enhanced through training, aiding, or automation. One approach for improving the 
overall level of human perfonnance in a task is to understand how proficient individuals 
perfonn that task. By studying in detail the general knowledge, specific information, and 
reasoning processes an expert uses, a model of the task can be constructed that exhibits 
some of the properties of the expert being modeled (Klein, Calderwood, & MacGregor, 
1989). 

It would be a mistake to select a knowledge elicitation strategy without first 
developing a perspective on expert perfonnance. Any technique will highlight some 
aspects of expertise and de-emphasize others. To interpret the results of a knowledge 
elicitation effort, it is necessary to appreciate the various aspects of proficient 
performance. The knowledge elicitor needs to understand both what is being captured 
and what is being missed. One class of knowledge necessary for expertise is explicit and 
objective knowledge. This class includes factual knowledge, if/then rules, and analytical 
procedures. A second component of expertise, contextual knowledge, is described as 
tacit knowledge, since it is resistant to being articulated. Contextual knowledge is 
viewed as the background of practices enabling experts to articulate if/then rules and 
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apply analytical procedures. Judgments of typicality are tacit since you don’t have to 
analyze a situation to detennine experienced similar cases in the past. A third aspect of 
expertise involves perceptual learning and the development of perceptual-motor feel. As 
skills are mastered, finer discriminations are made and tools come to be manipulated 
automatically. It is essential that knowledge elicitation methods include some means of 
representing the contribution made by tacit knowledge and by perceptual learning. If the 
knowledge elicitation method is insensitive to tacit knowledge, then it is easy to draw the 
mistaken conclusion that expert knowledge is sufficient for perfonning a task well 
(Klein, Calderwood, & MacGregor, 1989). 

The CDM is a retrospective interview strategy that applies a set of cognitive 
probes to actual nonroutine incidents that required expert judgment or decisionmaking. 
Once the incident is selected, the interviewer asks for a brief description. Then a 
semistructured format is used to probe different aspects of the subject task. The CDM 
offers some specific features that distinguish it from other knowledge elicitation 
strategies. The CDM, like all critical incident techniques, focuses on nonroutine cases. 
Incidents that are nonroutine or difficult are usually the richest source of data about the 
capabilities of highly skilled personnel. In a critical decision interview, questions always 
refer to a specifically recalled incident. We usually obtain more specific and useful 
information when we probe concrete and nonroutine events than when we ask about 
general rules and procedures. Probing in the CDM is not limited to responses that can be 
objectively anchored and verified. Questions sometimes require the decisionmakers to 
reflect on their own strategies and bases for decisions. Although specific questions are 
asked for each decision point, the order and wording can still follow the natural flow of a 
dialogue. Because interviewers have “heard the whole story” before probing begins, they 
are in a better position to adapt the timing and wording of questions to the specific case 
(Klein, Calderwood, & MacGregor, 1989). 

The procedures adopted for the critical decision interviews represent solutions to 
meeting the overall goals and practical considerations of a study. The basic procedures 
can be summarized in five steps: (Klein, Calderwood, & MacGregor, 1989) 

Select the incident . Incidents are selected that can illustrate nonroutine aspects 
of a domain. The goal is to probe for components that go beyond the general knowledge 
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and procedures that enable a competent individual to perform routine tasks; we want to 
study those components that might discriminate the true expert. We ask the 
decisionmaker to select an incident that was challenging and that, in his or her 
decisionmaking, might have differed from someone with less experience (Klein, 
Calderwood, & MacGregor, 1989). 

Obtain unstructured incident account . The interviewee is asked to describe the 
entire incident. This account proceeds without interruption by the interviewer, except for 
minor points of clarification. The procedure accomplishes several goals. First, it creates 
a context for understanding on the part of the interviewer. Second, the account serves to 
activate the interviewee’s memory of the event as a context for questioning. In addition, 
the procedure helps to achieve a high level of cooperation from the interviewee by 
establishing the interviewer as a listener rather than an interrogator (Klein, Calderwood, 
& MacGregor, 1989). 

Construct incident timeline . After the incident is related, the interviewer 
proceeds to reconstruct the account in the form of a timeline that establishes the sequence 
and duration of each event reported by the interviewee. The timeline serves to establish a 
shared awareness of the facts of the event from the interviewee’s perspective. 
Inconsistencies in the account may be detected and corrected on the basis of the timeline, 
and missing facts filled in (Klein, Calderwood, & MacGregor, 1989). 

Identify decision points . During the timeline construction, specific decisions are 
identified for further probing. A decision point is probed if the interviewee would agree 
that other reasonable courses of action were possible or that another expert (perhaps one 
with less or greater expertise) might have chosen differently (Klein, Calderwood, & 
MacGregor, 1989). 

Probe decision points . Different studies use different probes, depending on the 
objectives of the project. Questions to elicit the details of cue usage are almost always 
asked first as part of the timeline construction, and represent the current information that 
is likely to have been heeded at each event time. Prior knowledge is also probed. Probes 
about opinions are asked for each decision, both those that were actually deliberated and 
those that existed only hypothetically (Klein, Calderwood, & MacGregor, 1989). 
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Most interview sessions are planned to last for about two hours, but length can be 
adjusted to meet the complexity and time constraints of a given application. It is usually 
valuable to allow the interviewee to draw a diagram during the recounting of an incident 
and in response to specific probes. For many individuals the diagram serves as a 
necessary memory aid in reconstructing the key features of the incident. It also provides 
a common source of reference in communicating the participant’s perspective to the 
interviewer (Klein, Calderwood, & MacGregor, 1989). 

The needs of the specific research goal define the nature of the possible products 
derived from the CDM interview. One potential product is the Critical Cue Inventory 
(CCI). The CCI is a collection of all of the information and perceptual cues that are 
pinpointed in the protocols. Many of the probes in the critical decision interview are 
directed at gaining specific cues that were used in formulating a situation assessment or 
considering options. Many of these cues are not spontaneously mentioned by 
decisionmakers and do not result from asking very general questions. This is why 
cognitive probes are needed (Klein, Calderwood, & MacGregor, 1989). 

In the end, this entire process should result in a listing of auditory cues associated 
with the room clearing task as well as a method of prioritizing those cues for inclusion in 
a virtual environment representation of the task. 
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III. SUBJECT MATTER EXPERT INTERVIEW 


In order to gain insight into the auditory cues used in a room clearing operation, 
the first step was to recruit subject matter experts who were willing to be interviewed 
using the Critical Decision Method. The method used to recruit the SMEs was to send a 
mass e-mail to all of the current Naval Postgraduate School (NPS) students who fell into 
one of the following four categories: 

1. U.S. Army Special Operations officers 

2. U.S. Army Infantry officers 

3. U.S. Navy SEALs 

4. U.S. Marines. 

A total of 15 students volunteered and the field was narrowed down to eight SMEs 
based on the amount of experience that they actually had in room clearing operations. 
The final eight SMEs consisted of one U.S. Army Special Operations officer, one U.S. 
Army Infantry officer, one U.S. Navy SEAL, and five officers in the U.S. Marines. The 
average time in service of these eight SMEs was over 12 years. 

To conduct the interviews, individual meetings were held with each SME in a 
small 2-man study room at the Dudley Knox Library on the NPS campus. After a brief 
introduction, participants were told that the purpose of the interview was to obtain a 
detailed list of all the auditory cues used during an actual room clearing operation. Each 
SME was told that the interview results would be used to develop guidelines for a 
realistic virtual representation of the room clearing task and that the interviews were 
constructed using the Critical Decision Method of knowledge elicitation, which would 
result in a Critical Cue Inventory of auditory cues. The SMEs were given a description 
of the five steps of the CDM that would be used: 1) Select the incident. 2) Obtain 
unstructured incident account. 3) Construct incident timeline. 4) Identify decision 
points/phases. 5) Probe decision points/phases. Finally, it was explained that the last 
step would be to rank order the auditory cues for each phase of the operation by 
importance. 

At this point in the procedures, all eight of the SMEs indicated that they did not 
believe they would be able to recall what auditory cues they had used during their room 
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clearing operations. They stated that the task was primarily a visual task, that there were 
too many extremely loud noises to be able to differentiate between individual sounds, and 
that the excitement and adrenalin caused by conducting the task did not allow them to 
remember the auditory aspects of the task. All eight of the SMEs were convinced to 
continue by explaining to them that the CDM procedures would assist them in focusing 
their memory and attention on the auditory cues they had used. 

The first interview was used as a pretest of the interview procedures and questions 
that had been developed. The only change to the interview that resulted from the pretest 
was to rename the phases of the operation that would be used for in-depth probing into 
more easily understandable and distinguishable parts. 

A. SELECT THE INCIDENT 

All of the SMEs had a great deal of experience in different types of room clearing 
operations. To assist them in determining which particular incident to select for the 
interview, the SMEs were given a general description of some of the characteristics that 
they should use to select the incident. They were asked to bring to mind an occasion 
when they conducted a room clearing operation, if possible, when they used the Precision 
room clearing technique rather than the High Intensity technique. They were told that 
this should be an operation that offered minimal information prior to its execution; one in 
which there was the possibility of having noncombatants in the room to be cleared, the 
number and location of enemy forces was unknown, and if possible, one in which they 
were not familiar with the content and exact layout of the room to be cleared. They were 
asked to select an operation that was a challenge, that offered numerous auditory cues, 
and one in which their decisions and actions lead to a successful outcome. 

All of the SMEs were able to recall an actual room clearing operation that met 
these guidelines. 

B. OBTAIN UNSTRUCTURED INCIDENT ACCOUNT 

After the SMEs decided on a particular incident, they were asked to tell the story 
of that room clearing operation. They were told to give a brief description of the 
background leading to the mission and then explain everything they remembered from 
the time that they initiated movement until the end of the operation. They were told that 
the interviewer would listen and take a few notes and would only interrupt if clarification 
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was needed. It was explained that it might help to draw a map or sketch as they related 
the incident. 

All of the SMEs drew a map of the room and surrounding area relating to their 
operation. They went into extreme detail, relating their every action and thought 
throughout the process. The interviewer listened to the accounts and tried to obtain as 
great an understanding as possible as to the sequence of events and overall situation, as 
well as the tactics, techniques, and procedures used. 

C. CONSTRUCT INCIDENT TIMELINE 

After the SME was done relating his room clearing incident, the interviewer 
constructed a timeline of the operation using the information provided. This was done to 
ensure that the interviewer really did understand the overall situation and to demonstrate 
that level of understanding to the SME. The SMEs were asked to tell the interviewer if 
something was missed or if the sequence or duration of any event was misunderstood. 

D. IDENTIFY DECISION POINTS/PHASES 

Once the SME and interviewer agreed that the timeline was accurate, it was 
divided into four separate phases: 

1. Plan, prep, and movement to breach site. 

2. Creation of breach. 

3. Clearing of the room. 

4. Disengagement 

These phases provided the structure for asking specific probing questions aimed 
at determining the auditory cues involved in the task. 

E. PROBE DECISION POINTS/PHASES 

At this point, the process had provided a timeline of a specific mission broken 
down into four distinct phases. The SMEs were then asked a series of 13 questions about 
the auditory cues associated with each phase and how those auditory cues were used by 
the SME. The questions keyed on the auditory aspects of essential pieces of information 
that are critical for successfully completing a room clearing operation (See Appendix A). 

F. RANK-ORDER AUDITORY CUES 

The final step in the interview process was to have the SME consider all of the 
auditory cues he had mentioned during the probing questions and to prioritize those cues 
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according to the impact that they had on his ability to successfully complete each phase 
of the operation. The SMEs were reminded that the importance of an auditory cue could 
come from the fact that he was or was not able to hear something, or it could come from 
the fact that the enemy was or was not able to hear something. The SMEs prioritized 
each auditory cue for each phase of the operation. 

G. INTERVIEW RESULTS 

Each interview lasted between 2 and 3 1/2 hours and resulted in a great deal of 
specific infonnation and expert insight into room clearing auditory cues. Before the 
interviews began, all of the SMEs were skeptical about how much useful information that 
they would be able to recall relating to the auditory aspect of the task. At the end of the 
process, however, every one of the SMEs commented on how surprised they were at the 
level of detail they were able to reach regarding the presence and relative importance of 
the associated auditory cues. The structured interview built on the framework of the 
Critical Decision Method helped them to remember and focus on auditory cues that they 
had never considered before. 

At this point, the interview process had provided a complete list of auditory cues 
used by each SME during their particular room clearing operation, but it was still 
necessary to consolidate the results across all eight experts. In order to obtain overall 
results from each of the individual interviews, a weighting method was applied to each of 
the SME’s rank-ordered results. The Auditory Critical Cue Inventory depicted in Table 2 
awarded a specific auditory cue an overall score of 14 points for each time that a SME 
ranked that cue as the most important cue for a specific phase of the operation. A score 
of 13 points was given for each number 2 ranking, 12 points for each number 3 ranking, 
and so on. No points were given if the SME did not mention a particular auditory cue. 

The Auditory Critical Cue Inventory shows the final rank of each auditory cue by 
phase, the rank that each individual SME gave the auditory cue, and the overall score that 
each cue earned as a result of the weighting process. 
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RANK 






SME 




SCORE 


Phase 1: Movement to Breach Site 

1 

2 

3 

4 

5 

6 

7 

8 


1 

Radio comms with higher 

2 

1 

2 

1 

1 

3 

3 


92 

2 

Stealth/Sound of own unit 

3 

2 

3 

3 


4 

1 

3 

86 

3 

Enemy voice/activity 

1 

4 

4 



1 

5 

2 

73 

4 

Verbal comms within team 

4 



2 

2 

2 


1 

64 

5 

Background noise 


3 


4 

3 

5 

4 


56 

6 

Enemy weapons 



1 





4 

25 

7 

Radio comms within team 







2 


13 

8 

Supporting unit weapons 



5 






10 













Phase 2: Actions at Breach Site 










1 

Verbal comms within team 

1 



3 

3 

1 

1 

1 

80 

2 

Stealth/Sound of own unit 

3 

3 


4 

1 


4 

2 

73 

3 

Enemy voice/activity 


1 

3 

1 


3 



52 

4 

Background noise 


4 


5 

2 

5 



44 

5 

Radio comms with higher 

4 

2 


2 





37 

6 

Sound of successful breach 

2 

5 





2 


36 

7 

Sound of supporting units 





4 

2 


4 

35 

8 

Room acoustics from object thrown into room 



2 




3 


25 

9 

Enemy weapons 



4 



4 



22 

10 

Radio comms with supporting unit 







5 

3 

22 

11 

Friendly grenade 



1 






14 

12 

Friendly weapons 



5 






10 













Phase 3: Room Clearing 










1 

Verbal comms within team 

2 

2 

1 

2 

3 

1 

1 

1 

107 

2 

Enemy voice/activity 

1 

3 

2 

4 

2 

4 

4 

4 

96 

3 

Enemy weapons 

4 

1 

4 



5 

2 

3 

71 

4 

Friendly weapons 



3 


1 

3 

3 

2 

63 

5 

Stealth/Sound of own unit 


4 


1 

4 



7 

44 

6 

Radio comms with higher 


5 


3 



5 


32 

7 

Friendly grenade 






2 



13 

8 

Activity/movement of team 

3 








12 

9 

Radio comms within team 




5 





10 

10 

Movement of non-combatants 








5 

10 

11 

Background noise 




6 





9 

12 

Sound of supporting unit 






6 



9 

13 

Acoustics/echo 







6 


9 













Phase 4: Disengagement 










1 

Verbal comms within team 

1 

2 

1 

2 

3 


1 


80 

2 

Radio comms with higher 

2 

1 


1 

4 


4 


63 

3 

Enemy voice/activity 


3 

2 




3 


37 

4 

Enemy weapons 



3 


1 




26 

5 

Background noises 


4 



2 




24 

6 

Radio comms within team 







2 


13 

7 

Radio comms with supporting unit 

3 








12 

8 

Extraction platform 


5 







10 


Table 2. Room Clearing Auditory Critical Cue Inventory 
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IV. EVALUATION OF ROOM CLEARING SCENARIO 


The next step in determining which auditory cues need to be included in a virtual 
room clearing scenario was to allow the room clearing subject matter experts to evaluate 
the current status of the auditory cues that were already modeled in the America’s Anny: 
Operations. The purpose of evaluating the sound cues from an expert’s perspective was 
to determine if the auditory cues presented might lead to any negative transfer of training 
to the real-world execution of the task. Any inaccuracies in the cues that caused a player 
of the game to perform a task in a way that is not consistent with how an expert would 
actually conduct the task would lead to a negative training transfer issue. The game 
could be training a technique that would have to be retrained in the correct way in the real 
world. An analysis of the results obtained by evaluating the auditory cues in AA:0 could 
be applied to the auditory cues in any virtual representation of the room clearing task. 

In order to evaluate the room clearing auditory cues in AA:0, a hypothetical room 
clearing mission scenario was developed. This same mission was then presented to the 
SMEs in three different ways. Phase One had the SMEs use the same rank ordering 
technique of auditory cues that they used during the CDM interviews. The difference 
was that the scenario that they used for detennining the importance of the cues was a 
hypothetical mission that they had not actually conducted in real life. Phase Two 
consisted of showing the SMEs a digital reproduction of the mission used in Phase One 
that had actually been conducted and recorded in AA:0. The SMEs watched the mission 
unfold and listened to the auditory cues that were presented as if they were actually 
playing the game. The SMEs then answered a series of questions about the accuracy and 
appropriateness of the auditory cues as well as any auditory deficiencies that they 
noticed. In Phase Three, the SMEs actually played the AA:0 scenario that they had just 
watched and listened to in Phase Two. After playing the game, the SMEs discussed any 
potential training deficiencies that they expected would result from using the scenario as 
a training tool. 

A. EVALUATION SCENARIO 

The first step in creating the mission scenario to use for evaluating the AA:0 
auditory cues was to become familiar with all of the cues that were available in the game. 
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The goal was to have the recorded version of the scenario present as many auditory cues 
as possible for the SMEs to evaluate. 

Working with the AA:0 game developers, a hypothetical mission was created in 
which a four-man team was dropped off at a warehouse complex with one room being 
held by two enemy personnel. The four-man team was given a route of march through 
the courtyard of the warehouse complex as well as a breach point into the hostile room. 
Each man on the team was assigned a proposed route inside the room to reach his 
designated point of domination. Appendix B contains the Mission Briefing as well as a 
large scale map of the virtual warehouse complex showing the route of march leading to 
the room to be cleared and a smaller scale map of the room with the breach point and 
individual points of domination of each man inside the room. These basic maps had 
already been created as a level in the AA:0. 

The next step was to record the execution of the scripted mission being played in 
the game so that it could later be played back to the SMEs. This recording process took 
eight people and a great deal of rehearsal and coordination. Four AA:0 game developers 
each played the role of one of the friendly team members, two played the role of enemy 
combatants inside the room to be cleared, one ran the recording equipment, and the last 
served as overall coordinator of actions. Each player had certain actions to perform and 
phrases to say as the mission progressed. 

The point of view for the entire operation was from the 3 rd man in the team. The 
four-man team started at their initial location inside the warehouse complex. Each man 
charged his weapon. The first man (team leader) gave the verbal command “Squad, form 
up!” and each man replied in order with “Ready!” The team leader then commanded 
“Move out!” and the team moved in order along their designated route, through the open 
courtyard of the warehouse complex, to their stack position outside the room to be 
cleared. Once they reached their breach point, the team stopped momentarily and 
listened for any activity inside the room. There were two enemy players inside the room. 
One was silent and stationary and the other was moving, walking, and charging his 
weapon. The team leader whispered “Report in.” and the other three team members 
answered “Ready.” At this point the team leader moved forward to the door that was the 
designed breach point, prepared to throw a flask-bang grenade through the door, 
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announced “Grenade”, and threw the grenade into the room. The signal for the team to 
enter the room was the detonation of the grenade, and each man entered, moved along his 
route to his designated point of domination, and eliminated any enemy resistance in the 
room. The enemy players, while negatively affected by the flash-bang grenade, 
attempted to shoot the friendly team as they entered the room. Both enemy players were 
shot, and all four of the friendly team members reached their point of domination. 
Individual team members announced “Enemy spotted!” as they saw the enemy players 
and “Enemy down!” as they shot them. All four friendly team members announced 
“Area secure!” and the mission ended as the team leader commanded “Cease fire!” 

The mission was recorded on a DVD from the point of view of the third man in 
the team. The recording captured all the visual effects as well as the auditory cues. 
Voice commands, weapon sounds, individual movement noises, and background sounds 
were all included in the recording. 

B. PHASE ONE 

Phase One of the evaluation process did not use the DVD recording or even the 
AA:0 game. This phase consisted of the SMEs receiving a verbal mission briefing using 
paper copies of the scenario maps (see Appendix B). The verbal format was used to 
ensure that any aspects of the game itself did not influence this phase of the evaluation. 
Unlike the actual missions that were used for probing during the CDM interviews, this 
exercise was intended as an extension of the SMEs’ experience to a hypothetical room 
clearing mission. The SMEs were oriented to the maps and the mission was explained to 
them. Each SME understood that they were operating as the third man in the room 
clearing team. Just as in the CDM interviews, the mission was broken down into phases. 
This time, only the first three phases were used: movement to the breach site, actions at 
the breach site, and clearing of the room. Once each SME understood the mission and his 
particular part of the plan, they were given a list of all the auditory cues for the three 
phases of the operation and asked to rank-order the cues by importance as they had done 
during the interviews. 

The reason this phase was included in the evaluation process was to ensure that all 
the SMEs were approaching this exercise with the mindset that it was a real exercise. 
The evaluations that the SMEs were going to be asked to make in phases two and three, 
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needed to be comparisons between the real-world execution of the task and what the 
AA:0 game offered for auditory cues. The evaluations were not to be comparisons 
between AA:0 and what the SMEs might have seen or heard in other games or virtual 
environments. Phase one was intended to ensure a comparison to the real world. 

C. PHASE TWO 

After discussing the relative importance of potential auditory cues for the 
operation during the first phase, the SMEs were shown the DVD reproduction of the 
mission that had just been examined. The entire mission was replayed five times in order 
to allow the SMEs to concentrate on certain auditory cues. They examined the voice 
communications, individual and unit movement sounds, weapons effects, acoustics of the 
surrounding area, and background noises. After all the SMEs were satisfied that they had 
a good appreciation of the quality of the auditory cues presented in the mission, they had 
a discussion of the accuracy and appropriateness of the audio cues. The SMEs were 
reminded that any comparisons should be made to the real-world auditory cues that they 
had mentioned during the first phase. 

D. PHASE THREE 

The third phase of the evaluation process allowed the SMEs to actually play 
America’s Army: Operations. AA:0 game developers continued to assume the role of 
the enemy and the three remaining friendly team members, while the SMEs rotated 
through playing the role of the number three man on the team, just as they had seen on 
the recorded version of the mission. Putting the SMEs into the game was intended to 
give the players a more immersive perspective than just watching the recorded mission 
version. After playing the game scenario of the mission, the SMEs were allowed to play 
other levels involving different scenarios in the game in order to give them a deeper 
appreciation of the auditory cues available in the game. They used different weapon 
systems, operated in different environments (forest, beach, jungle, tunnel), listened to 
different types of grenades, manipulated the different types of voice communications 
(shout, whisper, and radio), and heard different background noises. Not all of the 
environments and noises that the SMEs experienced in these different levels of the game 
related directly to the scenario that we had developed, but playing the different levels 
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allowed the SMEs to experience the auditory cues that could be presented in a different 
room clearing scenario that might take place in a different environment. 

Once the SMEs were done experiencing the AA:0 audio cues first hand, they 
discussed their overall impressions of the game and its auditory cues as well as any real- 
world training deficiencies that they felt someone would have if the game was used as 
their sole training method. The SMEs were asked to focus their discussion on any 
potential negative training transfer issues that they would expect to see as a result of the 
game audio. 

E. EVALUATION RESULTS 

The results of rank-ordering the Critical Cue Inventory using the evaluation 
scenario are shown in Table 3. Although there are some minor changes to the ranking of 
some of the individual auditory cues from the CDM ranking, these changes are explained 
by the fact that in the hypothetical evaluation scenario, there were no supporting or 
overwatching units, and there was no radio communications with a higher headquarters. 
With the exception of those changes, the trend to rank a particular auditory cue high or 
low was almost identical to the results generated by the interviews. This indicates that 
the SMEs were able to extend their expertise to the hypothetical scenario and evaluate it 
in the same way that they evaluated their real-world training scenarios during the CDM 
interviews. 
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RANK 


RANK 



Phase 1: Movement to Breach Site 


Phase 3: Room Clearing 

1 

Enemy voice/activity 

1 

Verbal comms within team 

2 

Enemy weapons 

2 

Enemy voice/activity 

3 

Verbal comms within team 

3 

Enemy weapons 

4 

Stealth/Sound of own unit 

4 

Friendly weapons 

5 

Background noise 

5 

Activity/movement of team 

6 

Radio comms within team 

6 

Acoustics/echo 

7 

Radio comms with higher 

7 

Friendly grenade 

8 

Supporting unit weapons 

8 

Movement of non-combatants 



9 

Background noise 


Phase 2: Actions at Breach Site 

10 

Radio comms within team 

1 

Enemy voice/activity 

11 

Stealth/Sound of own unit 

2 

Verbal comms within team 

12 

Sound of supporting unit 

3 

Stealth/Sound of own unit 

13 

Radio comms with higher 

4 

Enemy weapons 


5 

Friendly grenade 

6 

Room acoustics from object thrown into room 

7 

Sound of successful breach 

8 

Friendly weapons 

9 

Background noise 

10 

Radio comms with supporting unit 

11 

Radio comms with higher 

12 

Sound of supporting units 


Table 3. Evaluation Scenario Critical Cue Inventory 


The discussions of the accuracy and appropriateness of the auditory cues 
presented in America’s Army; Operations revealed many insights as to what an expert 
expects to hear in a virtual training scenario. The largest criticism that the SMEs had 
with the auditory cues was the lack of emotional response that the cues created. The 
overwhelming majority of comments indicated that the audio cues need to be louder and 
need to have more of an emotional and physical impact on the user. Grenades detonating 
and weapons firing, especially in a confined area like inside a room, need to drown out all 
other sounds for a prolonged period. The SMEs agreed that the emotional aspect of 
entering a room containing hostile enemy soldiers would have a profound impact on your 
ability to hear. They felt that the amount of adrenalin produced by this type of mission 
would certainly make the team members breath hard, which would also muffle additional 
sounds. The SMEs mentioned an interesting point relating to the emotional aspect of the 
scenario created by the background noises. Background noises ranked relatively low in 
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importance on the Critical Cue Inventories, but during the discussions, the SMEs 
mentioned that one good way to increase the emotion and immersion of the user is to 
have more background noises. These background sounds need not relate to the actual 
room clearing mission, but must merely be accurate for the setting in which the mission 
takes place. The SMEs felt that creating an emotional immersion in the user was critical 
for accurate training, and that accurate background noises definitely increased their 
emotional level and immersion. The lack of background noises precluded an emotional 
immersion by all the SMEs. 

An individual’s voice in the game is another area that the SMEs felt needed 
additional work and could lead to negative training if cot corrected. Similar to the points 
mentioned above, the SMEs felt that it is critical to allow the users to display a wider 
range of emotion as they speak in the game. Verbal communications within the team 
members ranked the highest in importance during the CDM interviews. The SMEs 
agreed that the importance of these voice communications comes not only from what is 
said, but also from the way it is said. Whispers need to be softer, and yells, especially 
during the actual room clearing phase, need to be louder and convey the emotional 
aspects that would be expected with such a high-intensity activity. Additionally, the 
options of available voice commands needs to accurately reflect the current Standard 
Operating Procedures of the unit conducting the operation. The SMEs felt limited by the 
number of available voice commands that they could choose to say. 

The final area that the SMEs felt needed to be improved was the amount of noise 
that individuals and their equipment make as they move and operate in the virtual world. 
An individual’s ability to determine the level of stealth of himself and his unit ranked 
high in the Critical Cue Inventories and was of paramount importance during most phases 
of the CDM interviews. Although the virtual players in the scenario did make noise as 
their feet touched the ground, the SMEs were expecting a much more robust set of 
individual movement and equipment noises. They all felt that their packs and equipment 
would rattle somewhat, ammunition would rattle in plastic magazines, helmet snaps 
would make noises, their own heavy breathing would be hard to control, and their own 
footfalls would be much louder than what they experienced in the game. Not only are 
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these cues essential in detennining you own level of stealth, but also they are the means 
by which the friendly unit can gain information on any enemy movement or activity. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. SUMMARY 

This thesis has examined the issue of determining which auditory cues need to be 
presented in a virtual training environment. The task of conducting a Close Quarters 
Combat Room Clearing Operation is extremely complex, difficult, and dangerous. As 
increasing technology in the virtual environment domain allows for such tasks to be 
trained with these technologies, great care must be taken to avoid any negative training 
issues. 

The Critical Decision Method of knowledge elicitation allows investigators to use 
the great store of knowledge possessed by subject matter experts in order to improve 
training techniques. A structured interview, built on the framework of the Critical 
Decision Method, and administered to SMEs in the room clearing domain, resulted in a 
list of auditory cues, prioritized by importance, for each phase of the task. 

This prioritized list of auditory cues, or Critical Cue Inventory, was used by room 
clearing SMEs to evaluate the auditory cues that are currently being presented during 
room clearing scenarios in the game America’s Army: Operations. This expert 
evaluation included a discussion of potential negative training issues that could result 
from the game’s auditory cues. Any lessons learned from the example room clearing 
scenario in AA:0 can be extended to other attempts at representing the room clearing 
task in a virtual enviromnent. The scenario in AA:0 was merely used as one example of 
a virtual representation of a task. These lessons should be incorporated into any future 
attempts to create a virtual MOUT trainer or a virtual trainer of any complicated task. 

B. RECOMMENDATIONS 

All attempts at replicating a complex task in a virtual environment must involve 
an analysis of the audio cues associated with the real-world perfonnance of that task. 
Ideally, the importance of accurate auditory cues would be recognized early, and the 
analysis of those auditory cues would occur during the initial Cognitive Task Analysis 
used to define the task. This way, one analysis could be done that addressed all the cues 
used during the execution of the task, and a separate analysis of the auditory cues would 
not need to be conducted. The Critical Decision Method is just one of many ways to 
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assist subject matter experts in recalling and explaining the auditory cues associated with 
a task. Carefully thought out and worded probing questions can help the SMEs to 
detennine which of the auditory aspects of the task are most important to the successful 
execution of the task and can result in a rank-ordered Auditory Critical Cue Inventory. 
That Auditory CCI should determine the priority of auditory cues included in the virtual 
representation of the task. 

This process, applied to the Close Quarters Combat Room Clearing task, and 
evaluated through the sound cues presented in America’s Anny: Operations, resulted in a 
recommendation to improve three aspects of the auditory cues in the game. The first 
recommendation is to try to illicit more of an emotional response through the use of 
auditory cues. Loud sounds such as grenades and weapons firing should be louder and 
have more of an impact on the player, yells during combat should sound more emotional, 
the effect of adrenalin causing hard or heavy breathing should be modeled, and the 
emotional impact from simple background noises creating a sense of immersion are all 
critical to creating an emotional impact on the user. The second recommendation is to 
widen the range of options for verbal communications within the game. The phrases that 
virtual players are capable of saying must match the established procedures of real-world 
experts. Similar to the first recommendation, these voice communications need to be 
able to be said in a wider range of emotional ways. Whispers should be softer, yells 
should be louder, and the emotional state of the player making the verbal communication 
should be able to be discerned by those hearing his voice. The pre-recorded voice 
communications should be categorized by the overall emotional level expected for a 
particular operation. For example, while conducting actions at the breach site, stealth is 
of utmost importance to the room clearing team. During this phase the player should be 
able to select from a list of phrases that reflect that concern. Once the team enters the 
room, however, a menu of yelled or screamed voice communications should be used to 
try and demonstrate the user’s highly aroused state. The final recommendation pertaining 
to the auditory cues in a virtual room clearing operation, is to continue to add well- 
modeled audio cues for equipment used by the virtual players. Rucksacks and equipment 
should rattle in a realistic way, ammunition should make a sound when it is moved inside 
a magazine, and everyone close to a running player should hear footfalls. These sounds 
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are key to determining your own level of stealth and to detennining the activity of the 
enemy. 

By eliciting knowledge from domain experts and using their expertise to evaluate 
the accuracy and appropriateness of auditory cues in a training virtual environment, 
designers of training systems can ensure that the auditory cues presented in the virtual 
scenario will not lead to any negative transfer of training. 

C. FUTURE WORK 

Although this study concentrated on the auditory cues associated with a room 
clearing operation, the same procedures could be applied to any task that is to be 
represented and rained in a virtual environment. Any training conducted in a virtual 
environment has the potential to produce a negative training effect if all the aspects of the 
virtual training are not thoroughly investigated and their impact on the individual’s 
training analyzed. This detailed analysis of auditory cues should be conducted for every 
virtual training task. 

In the future, the same evaluation procedure used in this study should be applied 
to room clearing operations conducted in other virtual environments such as the Navy 
and Marine Corps project called VIRTE (Virtual Technologies and Experiments). 
VIRTE is being designed as a fully immersive combat trainer and its ability to accurately 
present the auditory cues associated with any training task will be critical to its ability to 
conduct quality training. 

Although all of the SMEs used during this study had received extensive room 
clearing training, they were all students at the Naval Postgraduate School at the time the 
interviews and scenario evaluation were conducted. Some of the SMEs had conducted 
room clearing operations as recently as three months prior to the interviews, but some 
had not conducted an operation in the past five years. While the Critical Decision 
Method allowed all the SMEs to recall a great deal of information relating to the auditory 
aspect of the operation that they selected, future work should include using experts that 
are currently assigned to units with a room clearing mission. This process would allow a 
comparison and evaluation of the SMEs ability to recall auditory cues used during room 
clearing operations that they had conducted in the past. 
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Another method of obtaining the auditory cues associated with the room clearing 
task would be to equip soldiers with recording devices as they conduct a live-fire room 
clearing training exercise. Subject matter experts could then use those recorded auditory 
cues to validate and possibly enrich the Critical Cue Inventory created from the Critical 
Decision Method interviews. 
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APPENDIX A. INTERVIEW 


A. INTRODUCTION 

Subject Number:_ 

My name is MAJ Tom Greenwald, a Simulations Operations Officer in the U.S. 
Army and a MOVES student at the Naval Postgraduate School. You were asked to 
participate in this interview because you are a subject matter expert in the domain of 
MOUT Close Quarters Combat Room Clearing Operations. I will use the information 
obtained during this interview to help me develop an experiment that will be used as my 
Master’s Thesis at the NPS. 

The purpose of my Thesis is to examine the auditory cues associated with a 
MOUT room clearing operation and determine which of those cues need to be included in 
a virtual environment representation of the operation, what sound cues are associated 
with the perfonnance of the actual task, and if the lack of certain auditory cues will result 
in a detriment of task perfonnance. 

This interview will focus on a time that you actually conducted a room clearing 
operation and will attempt to determine what auditory cues you had available to help you 
in conducting the task, what cues you used, what cues you did not use, and an ordering of 
the cues from most important to least important for each phase of the operation. 

This interview was constructed using the Critical Decision Method (CDM) of 
knowledge elicitation, which will result in a Critical Cue Inventory of auditory cues. The 
CDM is composed of five steps that we will use: 1) Select the incident. 2) Obtain 
unstructured incident account. 3) Construct incident timeline. 4) Identify decision 
points/phases. 5) Probe decision points/phases. Our sixth and final step will be to ra nk 
order the auditory cues for each phase of the operation by importance. 

B. BACKGROUND INFORMATION 


Before we begin I need to gather some background information. 

Age:_Service:_Years in service:_Branch/MOS: 

Number of years experience in MOUT Room Clearing Operations:_ 

C. SELECT THE INCIDENT 


I would like you to bring to mind an occasion when you conducted a room 
clearing operation. If possible, I would like it to be a time when you used the Precision 
room clearing technique rather than the High Intensity technique. This should be an 
operation that offered minimal information prior to its execution; one in which there was 
the possibility of having noncombatants in the room to be cleared, the number and 
location of enemy forces was unknown, and if possible, one in which you not familiar 
with the content and exact layout of the room to be cleared. Try to select an operation 
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that was a challenge to you, that offered numerous auditory cues, and one in which your 
decisions and actions lead to a successful outcome. 

D. OBTAIN UNSTRUCTURED INCIDENT ACCOUNT 

I would now like you to tell me the story of that room clearing operation. Give 
me a brief description of the background leading to the mission and then tell me 
everything you remember from the time you initiated movement until the end of the 
operation. I will listen and take a few notes and will only interrupt you if I need 
clarification. It may help you to draw a map or sketch as you go. 

E. CONSTRUCT INCIDENT TIMELINE 


Now I would like you to help me as I make a timeline of the operation. Tell me if 
I miss something or if I get the sequence or duration of any event wrong. 

F. IDENTIFY DECISION POINTS/PHASES 

Now I would like to divide our timeline that we just created into four separate 

phases: 

5. Plan, prep, and movement to breach site. 

6. Creation of breach. 

7. Clearing of the room. 

8. Disengagement. 

G. PROBE DECISION POINTS/PHASES 

Now we have our timeline of your mission broken down into four phases. I will 
now ask you a series of questions about each of these phases and the auditory cues that 
you had available and how you used them. 
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During Phase_ 

1. What auditory cues helped you detennine the answer to the following questions: 

a. Where am I (in respect to the urban area or my assigned sector)? 


b. Where are my soldiers? What is their current status/activity? 


c. Where are friendlies (adjacent and supporting units)? What is their current 
status/activity? 


d. Where is the enemy? What are the enemy’s capabilities? 


2. What was your primary concern during this phase? 


3. What were you primarily listening to or for? 


4. Did the actual execution of this phase differ from the plan? 


a. If it did differ, how were you able to recover? 


b. Did your ability to recover have anything to do with auditory cues? 


5. Was there any background noise? 


a. What was it? 
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b. Did it impact in any way on your ability to conduct the mission? 



6. What was the most important piece of infonnation that you did not have? 



a. Could you have received this information in the form of an auditory cue? 



b. What would have had to change in order for you to have received this cue? 



7. Name one thing that you wished you could hear but could not. 



8. Name one thing that you wished you could not hear. 



a. What negative impact did this unwelcome cue have? 



9. How important was the stealth of the friendly force during this phase? 



10. How much information do you think you were able to discern based on the 
acoustics of the room or area of operation? 



11. How much of an impact did extremely loud noises (weapons firing, grenades) 
have on your ability to successfully complete this phase? 
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12. How much of an impact did the auditory cues have on your ability to maintain 
situational awareness during this phase? 


13. Which auditory cues aided or detracted from your situational awareness during 
this phase? 


H. RANK ORDER AUDITORY CUES 


We will now review the auditory cues that you mentioned during the proceeding 
questions. When we are done I want you to prioritize those cues according to the impact 
that they had on your ability to successfully complete this phase of the operation. 
Remember that the importance of an auditory cue could come from the fact that you were 
or were not able to hear something, or it could come from the fact that the enemy was or 
was not able to hear something. 


1 . 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 

9. 

10 . 

11 . 

12 . 

13. 

14. 
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APPENDIX B. MISSION BRIEFING AND MAPS 


MISSION BRIEFING 

Orient everyone to maps 
Enemy Situation: 

There are an unknown number of enemy forces in the designated room. 

Enemy equipment and weapons are unknown. 

All other areas of the complex are secure. 

Friendly Situation: 

It is unknown if there are any non-combatants in the room. 

There are no supporting units. 

There are no overwatching units. 

You are the third man in a 4-man team. 

You are equipped with standard infantry weapons, an M-16, fragmentation 
grenades, and flash-bang grenades. 

Mission: 

Your 4-man team will move from the SP, along the designated route, to 
the point of entry into the room. Upon reaching the point of entry, the first man 
will deploy a flash-bang grenade through the open door. All team members will 
enter the room, reach your designated point of domination, and eliminate all 
enemy resistance. 

Coordinating Instructions: 

Your team has squad radios and internal voice communications available. 

Exercise as much stealth as possible until the flash-bang signal to enter the room. 
Try to gain as much information as possible about the enemy situation and any 
non-combatants in the room prior to entry. 

End of mission will occur when the room is deemed clear of all enemy. 



B 


LARGE SCALE SCENARIO MAP 



Figure 1. Large Scale Scenario Map 
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c. 


SMALL SCALE SCENARIO MAP 



Figure 2. Small Scale Scenario Map 
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APPENDIX C. PARTICIPANT CONSENT FORMS 


1. Introduction. You are invited to participate in a study exploring the auditory cues 
associated with a Close Quarters Combat Room Clearing Operation. This research is 
aimed at improving sound systems in virtual environments. . Your recorded data will be 
used in an effort to determine which cues need to be included in a virtual environment 
representation of the subject task. You will participate in one of two phases of this 
research: 

a. Phase One is a knowledge elicitation interview in which you will be asked a series 
of questions relating to the auditory cues you used to conduct an actual room 
clearing operation in the past. 

b. In Phase Two, you will be playing a scenario in America’s Army: Operations. 
After the scenario you will complete a questionnaire to indicate how the auditory 
cues presented impacted on your ability to complete the mission. 

2. Background Information. Data is being collected by the Naval Postgraduate School’s 
Human System’s Integration Laboratory for use in developing virtual environments. 

3. Procedures. If you participate in Phase One of this study, the researcher will ask you 
to recall and explain a time when you conducted a room clearing operation. You will be 
asked a series of questions relating to the auditory cues that you used during the conduct 
of that task. This entire task will take approximately 60 minutes. If you participate in 
Phase Two of this study, you will use the mouse and keyboard to play an America’s 
Army: Operations game scenario. You will then complete a written questionnaire. This 
entire task will take approximately 30 minutes. 

4. Risks and Benefits. The interview and experiments involve no risks to individuals, 
however, IF YOU FEEL UNCOMFORTABLE WITH THE INTERVIEW OR THE 
GAME SCENARIO AT ANY TIME, PLEASE INFORM THE EXPERIMENTER AT 
ONCE AND DO NOT PROCEED ANY FURTHER. The benefits to the participants 
will be to contribute to current research in advancing virtual environments and in human- 
computer interaction. 

5. Compensation. No tangible reward will be given. A copy of the results will be 
available to you at the conclusion of the experiment. 

6. Confidentiality. The records of this study will be kept confidential. No infonnation 
will be publicly accessible which could identify you as a participant. 

7. Voluntary Nature of the Study. If you agree to participate, you are free to withdraw 
from the study at any time without prejudice. You will be provided a copy of this form 
for your records. 
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8. Points of Contact. If you have any further questions or comments after the completion 
of the study, you may contact the research supervisor, Dr. Russell Shilling (831) 656- 
2543 shilling@cs.nps.navy.mil . 

9. Statement of Consent. I have read the above information. I have asked all questions 
and have had my questions answered. I agree to participate in this study. 


Participant’s Signature 


Date 


Researcher’s Signature 


Date 
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NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 MINIMAL 
RISK CONSENT STATEMENT 


Participant: 

VOLUNTARY CONSENT TO BE A RESEARCH PARTICIPANT IN: An Analysis 
of Audio Requirements for a Virtual Environment Close Quarters Battle Room Clearing 
Operation. 

1. I have read, understand and been provided "Information for Participants" that provides 
the details of the below acknowledgments. 

2. I understand that this project involves research. An explanation of the purposes of the 
research, a description of procedures to be used, identification of experimental 
procedures, and the extended duration of my participation have been provided to me. 

3. I understand that this project does not involve more than minimal risk. I have been 
informed of any reasonably foreseeable risks or discomforts to me. 

4. I have been infonned of any benefits to me or to others that may reasonably be 
expected from the research. 

5. I have signed a statement describing the extent to which confidentiality of records 
identifying me will be maintained. 

6. I have been informed of any compensation and/or medical treatments available if 
injury occurs and if so, what they consist of, or where further information may be 
obtained. 

7.1 understand that my participation in this project is voluntary, refusal to participate will 
involve no penalty or loss of benefits to which I am otherwise entitled. I also understand 
that I may discontinue participation at any time without penalty or loss of benefits to 
which I am otherwise entitled. 

8. I understand that the individual to contact should I need answers to pertinent questions 
about the research is Professor Russell Shilling, Principal Investigator, and about my 
rights as a research participant or concerning a research related injury. A full and 
responsive discussion of the elements of this project and my consent has taken place. 


Signature of Principal Investigator Date Signature of Volunteer Date 


Signature of Witness 


Date 
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PRIVACY ACT STATMENT 

NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
PRIVACY ACT STATEMENT 


1. Authority: Naval Instruction 

2. Purpose: ANALYZE THE AUDITORY CUES USED DURING A CLOSE 
QUARTERS COMBAT ROOM CLEARING OPERATION 

3. Use: Response data will be used for statistical analysis by the Departments of the Navy 
and Defense, and other U.S. Government agencies, provided this use is compatible with 
the purpose for which the information was collected. The Naval Postgraduate School in 
accordance with the provisions of the Freedom of Information Act may grant use of the 
information to legitimate nongovermnental agencies or individuals. 

4. Disclosure/Confidentiality: 

a. I have been assured that my privacy will be safeguarded. I will be assigned a control or 
code number, which thereafter will be the only identifying entry on any of the research 
records. The Principal Investigator will maintain the crossreference between name and 
control number. It will be decoded only when beneficial to me or if some circumstances, 
which are not apparent at this time, would make it clear that decoding would enhance the 
value of the research data. In all cases, the provisions of the Privacy Act Statement will 
be honored. 

b. I understand that a record of the information contained in this Consent Statement or 
derived from the experiment described herein will be retained pennanently at the Naval 
Postgraduate School or by higher authority. I voluntarily agree to its disclosure to 
agencies or individuals indicated in paragraph 3 and I have been informed that failure to 
agree to such disclosure may negate the purpose for which the experiment was 
conducted. 

c. I also understand that disclosure of the requested infonnation is voluntary. 


Signature of Volunteer Name, Grade/Rank, DOB 


Date 


Signature of Witness Date 
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